Exact Hypothesis Testing without Assumptions - New and Old Results not only for Experimental Game Theory1

This is an overview of tests for hypothesis testing that only rely on objective characteristics of the data generating process and do not rely on additional assumptions when making inference. In particular the methods are distribution-free, many apply to nonparametric settings. Only exact results are presented, results that can be proven for the given …nite sample. Asymptotic theory is not used. The reader is reminded how to construct con…dence intervals for medians. Existing methods for comparing binomial proportions are contrasted using formal criteria. Hidden assumptions underlying the Wilcoxon rank sum test and the Spearman rank correlation test are exposed. New tests are presented for analyzing levels, dispersion and association and for comparing levels, tests that are based on ordinal comparisons. We also present con…dence intervals for a single mean or variance, for testing equality of two means or of two variances and for testing for correlation. These methods apply when all possible outcomes belong to a known interval. This overview is particularly guided towards experimental game theory where small samples are the rule and there is a tradition for minimizing additional assumptions imposed.